﻿ Inteligenţă Arﬁcială Cursurile 3-4 Proiectul nostru Să învățăm din proiecte trecute Legarea entăților în texte și în afara lor Agenți care înțeleg situațiile din viața de zi cu zi a unui partener uman Modelarea unor manifestări ale conșinței Proiectul nostru • Un socialbot capabil să poate o conversație despre orice domeniu • The Alexa Amazon compeon – hps://developer amazon com/alexaprize/ contest-rules IBM Watson • hps://www youtube com/watch? v=WFR3lOm xhE • hps://www youtube com/watch? v=3G2H3DZ8rNc Asimov chatbot • hps://www chatbots org/chatbot/asimov/ Loebner compeon • hp://www aisb org uk/media/ﬁles/ LoebnerPrize2016/Mitsuku pdf Proiectele cursului de IA • 2003-2004: Simularea unui joc de fotbal • 2004-2005: Sistem de traducere automată – modelul de transfer • 2005-2007: PhiSociety – societăți primive de agenți inteligenți în compeție • 2007-2008: Casa inteligentă • 2008-2009: Avatar uman vorbitor • 2009-2010: Extragere de conținut semanc din texte • 2010-2012: Companion • 2012-2013: Let’s talk about books! • 2013-2014: MappingBooks: Let’s jump out of the book! • QuoVadis – Chasing enes and linking them (a project with the master students in Computaonal Linguiscs) • 2014-2015: MyDailyLife • 2015-2016: Modelarea conșinței MappingBooks: Evade from the book in the real world! QuoVadis: Linking enes in texts MappingBooks • A project proposal (submied in May 2013) in the naonal research projects contest (667 388 EUR); partners: UAIC-FII, University “Ștefan cel Mare” Suceava, Siveco – Bucharest: approved with the highest evaluaon score • A term project (Oct 2013 – Jan 2014) for 3rd year CS students in AI • More conference papers Linguiscs Linked Open Data (LLOD) - Develop techniques able to decipher the semanc context of texts - summaries (general, paral, focused on characters), - narrave lines (e g evoluon of senments between Vinicius and Ligia) - stac connecons between enes (e g genealogical trees), - stascs about enes (e g the dominant senments of Chrisans compared against those of Romans) 9 Linguiscs Linked Open Data (LLOD) - Generaon of ontologies from collecons of scienﬁc works - applicaons that “read” science books and formalise the concepts and its instances - Intelligent documentary search - Personalized assistants of a research acvity 10 Enty linking • Challenges in enty linking: – name variaons – ambiguies – absence • enty • link type 11 Linking enes internally 12 Linking enes externally 13 I like to read books and to travel… Going out of the book… Çelebi Mh , Maç Sk, Beyoğlu, Turkey to Çukur Cuma Cd, Beyoğlu, Turkey - Google Maps10/3/1310/3/13 8:13 PMKatip Çelebi Mh , Maç Sk, Beyoğlu, Turkey to Çukur Cuma Cd, Beyoğlu, Turkey - Google Maps 8:13 PMKatip Directions to Çukur Cuma Cd, Beyo!lu, Turkey 400 m – about 4 mins Walking directions are in beta Use caution – This route may be missing sidewalks or pedestrian paths Katip Çelebi Mh , Maç Sk, Beyo!lu, Turkey" Çukur Cuma Cd, Beyo!lu, Turkey" 1 Head southwest on Maç Sk toward Baltacı Çkgo 75 m About 47 secstotal 75 m These directions are for planning purposes only You may find that construction projects, traffic, weather, or other events may cause conditions to differ from the map results, and you should plan your route accordingly You must obey all signs or notices regarding your route Map data ©2013 Basarsoft 2 Turn right onto Turnacıba"ı Cdgo 28 m total 100 m 3 Turn left onto A!a Külhanı Sk (Altıpatlar Sk )go 130 m About 2 minstotal 240 m 4 Continue onto Çukur Cuma Cdgo 150 m About 1 mintotal 400 m Page 2 of 2https://maps google com/maps?f=d&source=s d&saddr=Maç+Sokak,+I…,288 55,2 369,37 281,0&layer=c&ei=OqVNUp3mE8nTtAaWr4CgCQ&pw=2 Page 1 of 2https://maps google com/maps?f=d&source=s d&saddr=Maç+Sokak,+I…,288 55,2 369,37 281,0&layer=c&ei=OqVNUp3mE8nTtAaWr4CgCQ&pw=2 I need help to remember all kinship relaons between characters 16 Characters in Forsyte Saga • The old Forsytes Ann, the eldest of the family Old Jolyon, the patriarch of the family, having made a fortune in tea James, a solicitor, married to Emily, a most tranquil woman Swithin, James's twin brother with aristocrac pretensions; a bachelor Roger, "the original Forsyte" Julia (Juley), a ﬂuery dowager; Mrs Sepmus Small Hester, an old maid Nicholas, the wealthiest in the family Timothy, the most cauous man in England Susan, the married sister • The young Forsytes Young Jolyon, Old Jolyon's arsc and free-thinking son, married three mes Soames, James and Emily's son, an intense, unimaginave and possessive solicitor, married to the unhappy Irene, who later marries Young Jolyon Winifred, Soames's sister, one of the three daughters of James and Emily, married to the foppish and lethargic Montague Dare George, Roger's son, a dyed-in-the-wool mocker Francie, George's sister and Roger's daughter, emancipated from God • Their children June, Young Jolyon's deﬁant daughter from his ﬁrst marriage; engaged to an architect, Philip Bosinney, who becomes Irene's lover Jolly, Young Jolyon's son from his second marriage; dies of enteric fever during the Boer Wars Holly, Young Jolyon's daughter from his second marriage, to June's governess Jon, Young Jolyon's son from his third marriage, to Irene, Soames's ﬁrst wife Fleur, Soames's daughter from his second marriage, to a French Soho shopgirl Annee; Jon's lover; later marries a baronet, Michael Mont Val, Winifred and Montague's son; ﬁghts in the Boer Wars; marries his cousin Holly Imogen, Winifred and Montague's daughter • Others Parﬁ, Old Jolyon's butler Smither, Aunts Ann, Juley and Hester's housekeeper Warmson, James and Emily's butler Bilson, Soames's housemaid 17 Prosper Profond, Winifred's admirer and Annee's lover 18 MappingBooks • A MappedBook is a book connected with locaons/events in the virtual and real world and sensive to the instantaneous locaon (as seized by the mobile/tablet) of a reader • The informaon made available could possibly be diﬀerent depending on the moment and the place of the reader MappingBooks • mul-dimensional mash-ups combining textual, geographical and temporal data • adequate presentaon to the reader • links sensive to: – the context of the menons in the book, – the moment the user iniates an access – the current locaon of the user • make heavy use of enty linking techniques • spot the book menons (persons and locaons) in the real and virtual world 20 SYNASC, UVT Timișoara, 27 Sept 2016 Aims 1) connect enes’ menons in the form of nominals (noun phrases) => one coreferenal chain corresponds to each enty; 2) no preliminary records about linked enes => the knowledge base evolves from scratch; 3) look specially for coreferenal (identy of enty menons) and geographical relaons (posion, distance, point-of, near, intersects, etc ); 4) texts under invesgaon: Geography manuals and traveling guides MappingBooks: what is it about? • “Understand” parts of a text • Recognise menons of persons and locaons • Recognise and crawl for real world enes • Know where I am • Seize what real world enes are in my proximity • Trace GoogleMaps paths, as described in the book • Fetch, process and make use of geo-data • Mix images with generated info • Display an aracve user interface • Client-server MappingBooks – an architecture Towards… live books • Muldimensional artefacts that combine textual, geographical, temporal, etc data • Evidence menons of persons, locaons… • Links sensible at: – the context of the menon in book – the locaon of the reader – the moment of the lecture – the personality and preferences of the reader 24 SYNASC, UVT Timișoara, 27 Sept 2016 Usage examples - I visit a city with the traveling guide in my hand - places of interest, routes, are reordered depending on my instantaneous posion 25 SYNASC, UVT Timișoara, 27 Sept 2016 Usage examples - I am a school boy, in the train going from Brașov to Sibiu… - if I open my tablet and head it towards the le side window of the train, I will see arrows showing the picks of the Făgăraș mountains, exactly as in the Geography manual 26 SYNASC, UVT Timișoara, 27 Sept 2016 Usage examples - I am in Paris for the 3rd me… - but only now my MB Lonely Planet guide signals me this temporary exhibion opened in the Pyramid 27 SYNASC, UVT Timișoara, 27 Sept 2016 MappingBooks is addressed to… • Youngsters, school people – could we bring back the book in their hands? • Teenagers, adventurers, travelers, lovers of excursions – socialize about places menoned in guides • Pensioners – socialize about common readings, cultural preferences • Researchers in Language Technology & Computaon Linguiscs – access to annotated linguisc resources • Owners of textual data (eding houses, media companies, tabloids) – sell beer their products • Local administraon, tourist agencies – ad on places of local tourisc interest 28 SYNASC, UVT Timișoara, 27 Sept 2016 The applicaon 1) Connects menons of enes (nominal groups) => one enty = a chain of coreferenal menons 2) The knowledge base does not include any apriory records about enes => starts from scratch 3) Idenﬁes geographical relaons (distances, posions, proximies, intersecons, etc ) 4) Texts, for the me being: geography manuals 29 SYNASC, UVT Timișoara, 27 Sept 2016 Stable user proﬁling • Data types for proﬁling – age, sex, educaon, profession, naonality, city and region address, traveling, music, readings and other cultural preferences, hobbies, etc • Sources: – connected social networks (Facebook, Twier, LinkedIn, Google+) where MB users are members – direct acquision at sign up ⇒ Make up speciﬁc stable “signatures” of users as aribute-value vectors 30 ICCCI-RUMOUR, Halkidiki, 28 Sept 2016 Instantaneous contexts (volale data collected during daily or sporadic use, quickly changing) • informaon seized by sensors or inferred: – the place where you are now, recent travels, the history of past travels • informaon accumulated on the server: – the book that you read now, recent readings, the history of your readings • informaon inpued by the user: – intenon to start a journey 31 ICCCI-RUMOUR, Halkidiki, 28 Sept 2016 Volale proﬁling • Contextual informaon ⇒ Make up speciﬁc volale “signatures” of users as aribute-value vectors Mixed proﬁle • Sum up the two types of data used in proﬁling 32 ICCCI-RUMOUR, Halkidiki, 28 Sept 2016 Accessibility of personal data • In accord with the trendy typology of accessibility of data adopted by current social media networks: – public – restricted to classes of friends – for personal use only 33 ICCCI-RUMOUR, Halkidiki, 28 Sept 2016 Matching • Apply scored vector matching techniques against other users’ signatures • Retain the best ranked matches 34 ICCCI-RUMOUR, Halkidiki, 28 Sept 2016 Networking Readers: enhance e-Books reading experience • Use semanc and geographical links to form communies – if books “subscripted for” declared visible => • current co-readers of B • co-readers of B – if “instantaneous or past locaon” declared visible => • current co-proximity of L • co-proximity of L • current co-track of T • co-track of T – any combinaons 35 ICCCI-RUMOUR, Halkidiki, 28 Sept 2016 Networking Readers: enhance e-Books reading experience • Easy to imagine other ways to form communies rooted in lectures – intersect common readings and aended places with levels of friendship reported by other social media, like Facebook or Twier – real-world events and enes menoned in a book associated with real-world locaons and parcular moments of the year/day 36 ICCCI-RUMOUR, Halkidiki, 28 Sept 2016 TA = Text Analytics NER = Name Entity Recognition AR = Augmented Reality EC = Entity Crowling DEV = Device Info RD = Relations Detection INT = Interfaces GEO = Geography RES = Resources M&T = Maps and Trajectories M&E = Management and Evaluation 37 SYNASC, UVT Timișoara, 27 Sept 2016 MappingBooks: acknowledgements • Undergraduate students from the Faculty of Computer Science in Iași, who in the ﬁrst term of the university year 2013-2014 have build the MB prototype as a term project in the AI course • Romanian Ministry of Educaon and Research for the grant PN-II-PT-PCCA-2013-4-1878 Partnership PCCA 2013 “MappingBooks - Intră în carte!”, July 2014 – December 2017 • Mihai Niculiță (Fac Geography) in UAIC – for rendering the maps in MB • Vivi Năstase for relevant ideas and realisaon of two ﬁgures • The other two partner instuons: SIVECO – Bucharest and “Ștefan cel Mare” University of Suceava 38 ICCCI-RUMOUR, Halkidiki, 28 Sept 2016 QuoVadis: Chasing enes and links in books 39 SYNASC, UVT Timișoara, 27 Sept 2016 hps://nlptools info uaic ro The ‘QuoVadis’ corpus 40 SYNASC, UVT Timișoara, 27 Sept 2016 A corpus semanc enes and relaons • Type of enes: – persons – gods – groups of persons and gods – body parts • Remanc relaons among enes of these types 41 SYNASC, UVT Timișoara, 27 Sept 2016 Entities • individuals (Marcus Vinicius, Lygia), groups (the Christians, the soldiers) and classes (the emperor); • syntactic realisation: NPs (determiners – a soldier, adjectives – young patrician, complement PPs included – the son of one consul; but no relative clauses; • included entities for Romanian language: [te] [iubesc; REALISATION=INCLUDED], Marcus vs 12 [I] love [thee], Marcus 21 • recursive referential expressions: [the adherents [of Christ]] are praying… 21 42 Relations • Anaphoric relations: co-referential; • Non-anaphoric relations: – kinship; – affective; – social 43 SYNASC, UVT Timișoara, 27 Sept 2016 Anaphoric relations • coref • coref-interpret • member-of, has-as-member (inverse) • isa, class-of (inverse) • part-of, has-as-part (inverse) • subgroup-of, has-as-subgroup (inverse) • has-name, name-of (inverse) Example: [Lygia] was unable to answer, for weeping seized [her] anew Acte gathered 12 [the maiden] to her bosom, and strove to calm [her] excitement 34 coref ; coref-interpret ; coref 44 SYNASC, UVT Timișoara, 27 Sept 2016 Kinship relations • parent-of • child-of (inverse of parent-of) • grandparent-of and grandchild-of (inverse) • sibling (symmetrical) • ant-uncle-of, nephew-of (inverse relation) • cousin-of (symmetrical) • spouse-of (symmetrical) • unknown Example: "Pardon me, Lygia For me thou art [ [of a king]] and [ [of Plautius]] “ 43 child-of ; child-of 45 SYNASC, UVT Timișoara, 27 Sept 2016 Social relations • superior-of • inferior-of • in cooperation-with • colleague-of • in competition-with • opposite-to Example: [Petronius]…but to [his] misfortune [he] 123 [Cæsar himself], hence [he] roused [his] jealousy 456 in competition-with ; coref ; coref ; coref 46 SYNASC, UVT Timișoara, 27 Sept 2016 Affective relations • love • loved-by • hate • hated by • upset • friendship • worship Example: Vinicius entered Lygia's dungeon and remained there till daylight…Both changed by degrees into sad souls with [each] [other] 12 rec-love 47 SYNASC, UVT Timișoara, 27 Sept 2016 A complex example cui i-ar fi putut trece prin minte că [un patrician], [nepot şi 1 [fiu de [consuli]]], ar putea să se găsească printre gropari 432 Besides, into whose head could it enter that [a patrician], [the 1 grandson [of one consul]], [the son [of another]], could be 5276 found among servants, corpse-bearers RO: coref , kinship:grandchild-of ; kinship:child-of ; EN: coref , coref , kinship:grandchild-of , kinship:child-of 48 SYNASC, UVT Timișoara, 27 Sept 2016 Relaons • Anaphoric: coref John met Maria on the ski slope He raced her anafor antecedent 49 SYNASC, UVT Timișoara, 27 Sept 2016 Relaons • Anaphoric: coref John met Maria on the ski slope He raced her anafor antecedent 50 SYNASC, UVT Timișoara, 27 Sept 2016 Arguments and triggers in relaons • Kinship: parent-of … their father … source desnaon trigger 51 SYNASC, UVT Timișoara, 27 Sept 2016 Arguments and triggers in relaons • Social: inferior-of Cesar’ s principal courers … trigger desnaon source 52 SYNASC, UVT Timișoara, 27 Sept 2016 Arguments and triggers in relaons • Aﬀecve: worship Lygia dropped on her knees to implore someone else trigger desnaon source 53 SYNASC, UVT Timișoara, 27 Sept 2016 Enes Petroniu… Vinicius was the son of his oldest sister , who years before had married his father , a man of consular dignity from the me of Tiberius 54 SYNASC, UVT Timișoara, 27 Sept 2016 Anaphoric relaons: coref Petroniu… Vinicius was the son of his oldest sister , who years before had married his father , a man of consular dignity from the me of Tiberius 55 SYNASC, UVT Timișoara, 27 Sept 2016 Anaphoric relaons: coref Petroniu… Vinicius was the son of his oldest sister , who years before had married his father , a man of consular dignity from the me of Tiberius 56 SYNASC, UVT Timișoara, 27 Sept 2016 Anaphoric relaons: coref Petroniu… Vinicius was the son of his oldest sister , who years before had married his father , a man of consular dignity from the me of Tiberius 57 SYNASC, UVT Timișoara, 27 Sept 2016 Anaphoric relaons: class-of Petroniu… Vinicius was the son of his oldest sister , who years before had married his father , a man of consular dignity from the me of Tiberius 58 SYNASC, UVT Timișoara, 27 Sept 2016 Kinship relaons: sibling Petroniu… Vinicius was the son of his oldest sister , who years before had married his father , a man of consular dignity from the me of Tiberius 59 SYNASC, UVT Timișoara, 27 Sept 2016 Kinship relaons: child-of Petroniu… Vinicius was the son of his oldest sister , who years before had married his father , a man of consular dignity from the me of Tiberius 60 SYNASC, UVT Timișoara, 27 Sept 2016 Kinship relaons: parent-of Petroniu… Vinicius was the son of his oldest sister , who years before had married his father , a man of consular dignity from the me of Tiberius 61 SYNASC, UVT Timișoara, 27 Sept 2016 Kinship relaons: spouse-of Petroniu… Vinicius was the son of his oldest sister , who years before had married his father , a man of consular dignity from the me of Tiberius 62 SYNASC, UVT Timișoara, 27 Sept 2016 Social relaons: inferior-of Petroniu… Vinicius was the son of his oldest sister , who years before had married his father , a man of consular dignity from the me of Tiberius 63 SYNASC, UVT Timișoara, 27 Sept 2016 General stascs over the corpus • 7,281 sentences • 146,822 tokens, punctuaon included • 171,029 tokens summed up under all relaons • 24,636 enty menons • 22,301 referenal relaons • 755 AKS relaons (Aﬀecve + Kinship + Social) • 752 triggers 64 SYNASC, UVT Timișoara, 27 Sept 2016 Example: aﬀecve relaons love and worship 65 SYNASC, UVT Timișoara, 27 Sept 2016 Example: aﬀecve relaons fear-of and hate 66 SYNASC, UVT Timișoara, 27 Sept 2016 Vinicius’ links with other characters 67 SYNASC, UVT Timișoara, 27 Sept 2016 Semanc relaons involving Vinicius 68 SYNASC, UVT Timișoara, 27 Sept 2016 căsătorise Marcus cu Vinicius TYPE="parent-of"> era tatăl TYPE="child-of"> acestuia ﬁul TYPE="sibling-of"> surorii TYPE="inferior-of"> sale o consul mai pe mari vremea lui ta o Tiberiu n , n TYPE="spouse-of"> A care REFERENTIAL> , cu REFERENTIAL> ani în urmă , of" /REFERENTIAL> se 69 QuoVadis: thanks • Students in Master of Computaonal Linguiscs, series 2012-2014 – annotaon • Anca Bibiri, Cătălina Mărănduc, Daniela Gîfu – coordinaon, validaon • Paul Diac, Andrei Scutelnicu, Mihaela Colhon – programming 70 SYNASC, UVT Timișoara, 27 Sept 2016 Proiectul IA al seriei 2014-2015 MyDailyLife • Un agent capabil să “înțeleagă” situațiile în care se aﬂă Masterul său, să le transpună în text și să conducă un dialog relav la ele Proiectul IA al seriei 2014-2015 MyDailyLife • Un agent capabil să “înțeleagă” situațiile în care se aﬂă Masterul său, să le transpună în text și să conducă un dialog relav la ele Ce trebuie să facă sistemul? • Să “șe”, în cât mai multe momente ale zilei: – ce fac – unde mă aﬂu – cu cine mă întâlnesc – despre ce vorbesc – de ce fac anumite lucruri • Să ﬁe capabil să formeze fraze care exprimă ce am făcut • Să ﬁe capabil să poarte un dialog – despre ziua de azi, cea de ieri – cea de mâine (?) Cui se adresează? Persoanelor în pericol de boli degenerave ale creierului (Alzheimer etc ) Un graf al situațiilor – ierarhie de clase alive sleeping awake at home moving in acvity not moving in theater in park shopping driving walking jogging driving in town driving outside Module 1 LOCALISATION 5 INFERENCES 2 MOVEMENT 6 BEHAVIOUR 3 FACES 7 TEXT GENERATION 4 SOUNDS&VOICES 8 QUESTION-ANSWERING 9 INTERFACES 10 CLIENT-SERVER 11 STANDARDS AND EVALUATION System descripon • A client-server architecture • 3 types of modules: • interpreng primary data from my mobile • doing inferences and learning my behaviour • generang text and engaging dialogs day-me sensing evening acvies MyDailyLife – schema generală Acvies Behaviour graph sensors Text generaon Diary repr Localisaon Movement Inferences Diary Queson Faces answering MyDiary Sounds& voices Interfețe și comunicații MyDiary GPS Localisaon COMM activități zilnice vibrații Movement COMM Inferences imagini COMM Faces sunete Sounds/COMM Voices Interfețe și comunicații Inferences SIT Text Diary SIT generaon repr query QUE NL MyDiary NL Queson Diary answering dialogul de seară Interfețe și comunicații Inferences SIT Text Diary SIT generaon repr query QUE NL MyDiary NL Queson Diary answering dialogul de seară Exemplu • TIME=7:30, PROXIMITY = bedroom, CLOCK = awake me, SOUND = awake ringing, GG:LIGHT = intensity raise, WiFi WAVE = movement, ACCELERATION = zero, GPS = null, GEO POSITION = home(remember last posion) • MyDailyLife output: Astă dimineață am fost trezit de alarmă la ora 7:30 în dormitorul meu 2015-2016: Modelarea unor manifestări ale conșinței • Intrări – 3 puri de canale: – textual (un text) – acusc (inregistrări vocale) – vizuale (imagini) • Ieșire: reacții (emoții) similar celor umane Producerea materialelor de învățare • Prin explicitarea, pe eșanoanele alese, a: – emoțiilor – dorințelor – gândurilor pe pare le manifestați voi inșivă la cirea/ascultarea/ vederea intrărilor • Veți căuta apoi seturi de intrări similare (care vă produc reacții similare) formând asel seturi de antrenament • Notați manifestările voastre emove pe intrările alese într-un standard (XML), pe care trebuie să-l propuneți Atudinea față de personaje • Ați cit deja romanul “Quo Vadis”… • Ați notat pe manșeta foilor de hâre (ecranului) senmentele pe care le încercați față de diverse personaje în diferite momente din derularea acțiunii • Inventați un stardard (XML) de notare a atudinii voastre față de personaje în funcție de evoluția evenimentelor: atudinile lor, acțiunile lor, gândurile lor Adnotări aplicate textului • Module care crează marcaje: – tokenizator – lemazator – POS-tagger La apelul FDG-parserului – sentence splier – sintacc-parser (FDG) – NP-chunker – NER (name enty recogniser) Adnotări existente în corpusul QV – anaphora resolver – event-detector (se bazează pe adnotările obținute în toate Acest modul trebuie nivelurile anterioare) creat de voi 